# Knowledge Distillation Optimization

A 7-billion-parameter diffusion model jointly developed by Freepik and Fal, built through knowledge distillation with fast generation and efficient memory usage characteristics

Ultravox is a multimodal speech large language model based on Llama3.1-8B-Instruct and Whisper-small, capable of processing both speech and text inputs.

Ultravox is a multimodal voice large language model built upon Llama3.3-70B and Whisper, supporting both voice and text inputs, suitable for scenarios like voice agents and translation.

An 8-layer embedding model distilled from BAAI/bge-m3, achieving 2.5x speed improvement while maintaining retrieval performance

Ultravox is a multimodal model based on Mistral-Nemo and Whisper, capable of processing both speech and text inputs, suitable for tasks like voice agents and speech translation.

Ultravox is a multimodal speech large language model, built upon the pre-trained Llama3.1-70B-Instruct and whisper-large-v3-turbo backbones, capable of receiving both speech and text as inputs.

Ultravox is a multimodal speech large language model built on Llama3.1-8B-Instruct and whisper-large-v3-turbo, capable of processing both speech and text inputs.

An improved Polish re-ranking model based on sdadas/polish-roberta-large-v2, trained with RankNet loss function and supports Flash Attention 2 acceleration

Llama3.1-Neo-1B-100w is an efficient language model pruned to 1.4B parameters from Meta-Llama-3.1-8B-Instruct and fine-tuned using the LLM-Neo method (combining LoRA and knowledge distillation). The training data consists of 1 million samples from BAAI/Infinity-Instruct.

A Bangla sentence embedding model fine-tuned from stsb-xlm-r-multilingual, supporting sentence similarity calculation and semantic search

A dual-encoder dot-product scoring architecture based on DistilBert, trained on the MSMARCO-Passage dataset with balanced topic-aware sampling, suitable for dense retrieval and candidate set re-ranking

MiniLMv2 is a lightweight language representation model developed by Microsoft, achieving efficient performance through knowledge distillation technology.

DistilBERT-based dense retrieval model trained with knowledge distillation, suitable for passage re-ranking and direct retrieval tasks